Stopping Times

Stopping times

Let \((\Omega, \sF, P)\) be a probability space. Consider a random process \(X:\Omega\to\sX^T\) defined on this probability space with state space \(\sX \subseteq \R\) and ordered index set \(T\subseteq \R\) considered as time.

Definition 1. A collection of event spaces denoted \(\sG_{\bullet} \triangleq (\sG_t \subseteq \sF: t \in T)\) is called a filtration if \(\sG_s \subseteq \sG_t\) for all \(s \le t\).

Remark 1. For the random process \(X:\Omega\to\sX^T\), we can find the event space generated by all random variables until time \(t\) as \(\sF_t\triangleq \sigma(X_s, s \le t)\). The collection of event spaces \(\sF_\bullet \triangleq (\sF_t: t \in T)\) is a filtration.

Definition 2. The natural filtration associated with a random process \(X:\Omega \to \sX^T\) is given by \(\sF_\bullet \triangleq (\sF_t: t\in T)\) where \(\sF_t \triangleq \sigma(X_s, s\le t)\).

Remark 2. For a random sequence \(X: \Omega \to \sX^\N\), the natural filtration is a sequence \(\sF_\bullet = (\sF_n: n \in \N)\) of event spaces \(\sF_n \triangleq \sigma(X_1, \dots, X_n)\) for all \(n \in \N\).

Remark 3. If the random sequence \(X\) is independent, then the random sequence \((X_{n+j}: j \in \N)\) is independent of the event space \(\sigma(X_1, \dots, X_n)\).

Example 3. For a random walk \(S\) with step size sequence \(X\), the natural filtration of the random walk is identical to that of the step size sequence. That is, \(\sigma(S_1, \dots, S_n) = \sigma(X_1, \dots, X_n)\) for all \(n \in \N\). This follows from the fact that for all \(n \in \N\), we can can write \(S_j = \sum_{i=1}^jX_i\) and \(X_j = S_j - S_{j-1}\) for all \(j \in [n]\). That is, there is a bijection between \((X_1, \dots, X_n)\) and \((S_1, \dots, S_n)\).

Definition 4. A random variable \(\tau: \Omega \to T\) is called a stopping time with respect to a filtration \(\sF_\bullet\) if

1. the event \(\tau^{-1}(-\infty, t] \in \sF_t\) for all \(t \in T\), and

2. the random variable \(\tau\) is finite almost surely, i.e. \(P\set{\tau < \infty} = 1\).

Remark 4. Intuitively, if we observe the process \(X\) sequentially, then the event \(\set{\tau \le t}\) can be completely determined by the observation \((X_s, s \le t)\) until time \(t\). The intuition behind a stopping time is that its realization is determined by the past and present events but not by future events. That is, given the history of the process until time \(t\), we can tell whether the stopping time is less than or equal to \(t\) or not. In particular, \(\E[\SetIn{\tau \le t}\given \sF_t]=\SetIn{\tau \le t}\) is either one or zero.

Definition 5 (First hitting time). For a process \(X:\Omega\to\sX^T\) and any Borel measurable set \(A \in \sB(\sX)\), we define the first hitting time \(\tau_X^A: \Omega \to T\cup\set{\infty}\) for the process \(X\) to hit states in \(A\), as \(\tau_X^A \triangleq \inf\set{t \in T: X_t \in A}.\)

Example 6. We observe that the event \(\set{\tau_X^A \le t} = \set{X_s \in A \text{ for some }s \le t} \in\sF_t\) for all \(t \in T\). It follows that, if \(\tau_A\) is finite almost surely, then \(\tau_A\) is a stopping time with respect to filtration \(\sF_\bullet\).

Proposition 7. For a random sequence \(X: \Omega \to \sX^\N\), an almost sure finite discrete random variable \(\tau: \Omega \to \N\cup\set{\infty}\) is a stopping time with respect to this random sequence \(X\) iff the event \(\set{\tau = n} \in \sigma(X_1, \dots, X_n)\) for all \(n \in \N\).

Proof. Proof. From Definition [defn:StoppingTimeGen], we have \(\set{\tau = n} = \set{\tau \le n}\setminus\set{\tau \le n-1} \in \sigma(X_1, \dots, X_n)\). Conversely, from the theorem hypothesis, it follows that \(\set{\tau \le n} = \cup_{m=1}^n\set{\tau = m} \in \sigma(X_1, \dots, X_n)\). ◻

Example 8. Consider a random sequence \(X: \Omega \to \sX^\N\), with the natural filtration \(\sF_\bullet\), and a measurable set \(A \in \sB(\sX)\). If the first hitting time \(\tau_X^A:\Omega\to\N\cup\set{\infty}\) for the sequence \(X\) to hit set \(A\) is almost surely finite, then \(\tau_X^A\) is a stopping time. This follows from the fact that \(\set{\tau_X^A = n} = \cap_{k=1}^{n-1}\set{X_k \notin A}\cap\set{X_n \in A} \in \sF_n\) for each \(n \in \N\).

Definition 9. Consider a random process \(X:\Omega\to \sX^{\R_+}\) with discrete state space \(\sX\subseteq\R\). For each state \(y \in \sX\), we define \(\tau_X^{\set{y},0} \triangleq 0\) and inductively define the \(k\)th hitting time to a state \(y\) after time \(t = 0\), as

Remark 5. We observe that \(\set{\tau_X^{\set{y}, k} \le t} \in \sF_t\) for all times \(t \in \R_+\). Hence if \(\tau_X^{\set{y}, k}\) is almost surely finite, then it is a stopping time for the process \(X\).

Definition 10. For a discrete valued random sequence \(X:\Omega\to\sX^\N\), the number of visits to a state \(y \in \sX\) in first \(n\) time steps is defined as \(N_y(n) \triangleq \sum_{k=1}^n\SetIn{X_k = y}\) for all \(n \in \N\).

Remark 6. We observe that \(N_y:\Omega\to\Z_+^{\Z_+}\) is a random walk with the Bernoulli step size sequence \((\SetIn{X_k = y}: k \in \N)\). Further, \(\tau_X^{\set{y}, k} = \tau_{N_y}^{\set{k}} = \inf\set{n \in \N: N_y(n) = k}\). We also observe that \(\{N_y(n) \le k\} = \{\tau_y^{(k+1)} > n\}\) and \(\{N_y(n) = k\} = \{\tau_y^{k} \le n < \tau_y^{(k+1)}\}\).

Remark 7. We observe that the number of visits to state \(y\) in first \(n\) steps of \(X\) is also given by This implies that \(N_y(n)+1\) is the first hitting time to set of states \(\set{n+1, n+2, \dots }\) for the increasing random sequence \((\tau_X^{\set{y},k}:k \in \N)\).

Lemma 11 (Wald’s Lemma). Consider a random walk \(S: \Omega \to \R^{\Z_+}\) with step-sizes \(X: \Omega \to \R^\N\) having finite \(\E\abs{X_1}\). Let \(\tau\) be a finite mean stopping time with respect to this random walk. Then,

Proof. Proof. Recall that the event space generate by the random walk and the step-sizes are identical. From the independence of step sizes, it follows that \(X_n\) is independent of \(\sigma(X_0, X_1, \dots, X_{n-1})\). Since \(\tau\) is a stopping time with respect to random walk \(S\), we observe that \(\set{\tau \ge n} = \set{\tau > n-1} \in \sigma(X_0, X_1, \dots, X_{n-1})\), and hence it follows that random variable \(X_n\) and indicator \(\SetIn{\tau \ge n}\) are independent and \(\E[X_n\SetIn{\tau \ge n}] = \E X_1 \E\SetIn{\tau \ge n}\). Therefore, We exchanged limit and expectation in the above step, which is not always allowed. We were able to do it by the application of dominated convergence theorem. ◻

Corollary 12. Consider the stopping time \(\tau_S^{\set{i}} \triangleq \min\set{n \in \N: S_n = i}\) for an integer random walk \(S: \Omega \to \Z_+^{\Z_+}\) with step size sequence \(X: \Omega \to \Z^\N\). Then, the mean of stopping time \(\E \tau_S^{\set{i}} = {i}/{\E X_1}\).

Proof. Proof. This follows from the Wald’s Lemma and the fact that \(S_{\tau_i} = i\). ◻

Properties of stopping time

Lemma 13. Let \(\tau_1,\tau_2\) be two stopping times with respect to filtration \(\sF_\bullet\). Then the following hold true.

1. \(\min \set{\tau_1,\tau_2}\) is a stopping time.

2. If \(T\) is separable, then \(\tau_1+\tau_2\) is a stopping time.

Proof. Proof. Let \(\tau_1,\tau_2\) be stopping times with respect to a filtration \(\sF_{\bullet} = (\sF_t : t \in T)\).

1. Result follows since the event \(\set{ \min \set{ \tau_1,\tau_2} > t} = \set{\tau_1 > t} \cap \set{\tau_2 > t} \in \sF_t\).

2. A topological space is called separable if it contains a countable dense set. Since \(\R_+\) is separable and ordered, we assume \(T = \R_+\) without any loss of generality. It suffices to show that the event \(\set{\tau_1+\tau_2 \le t} \in \sF_t\) for \(T = \R_+\). To this end, we observe that

 ◻

Strong independence property and applications

Theorem 14 (Strong independence property). Let \(X: \Omega\to\sX^{\R_+}\) be an independent random process with natural filtration \(\sF_\bullet\), and \(\tau:\Omega\to\R_+\) a stopping time with respect to \(\sF_\bullet\), then \((X_{\tau+s}: s \in \R_+)\) is independent of history \((X_s: s \le \tau)\).

Definition 15. We can define the \(k\)th return time to state \(y\) for the random process \(X:\Omega\to\sX^{\R_+}\) as the interval between two successive visits to state \(y\), that is for all \(k \in \N\)

Remark 8. We observe that \(\tau_X^{\set{y}, k} = \sum_{j=1}^kH_X^{\set{y}, j}\) for all \(k \in \N\). That is, the hitting times sequence \((\tau_X^{\set{y},k}: k \in \N)\) is a random walk with step-size sequence being return times \((H_X^{\set{y},j}: j \in \N)\). Therefore, if \(H_X^{\set{y}, j}\) is almost sure finite for all \(j \in [k]\), then the finite sum \(\tau_X^{\set{y}, k}\) is almost sure finite for all \(k\in \N\).

Remark 9. From the bijection between hitting and return times, we observe that \(\sigma(\tau_X^{\set{y}, k)}: k \in [n]) = \sigma(H_X^{\set{y}, j}: j \in [n])\). Recall that \(\tau_X^{N_y(n)+1} > n\) by definition. If passage times are independent and from second passage time, then it follows from the Wald’s Lemma that

Example 16. We also observe that the \(k\)th hitting time to \(\set{1}\) by a Bernoulli step size sequence \(X:\Omega\to\set{0,1}^\N\) is the first hitting time to \(\set{k}\) by random walk \(S:\Omega\to\Z_+^{\Z_+}\). That is,

Lemma 17. For an Bernoulli random sequence \(X:\Omega\to\set{0,1}^\N\) with \(\E X_1 \in (0,1)\), the \(k\)th hitting time to state \(1\) is a stopping time, and \(\tau_X^{\set{1}, k} = \sum_{i=1}^kY_i\) where \(Y:\Omega\to\N^\N\) is an random sequence distributed identically to \(\tau_X^{\set{1}}\) .

Proof. Proof. When the Bernoulli step size sequence \(X\) is with \(\E X_1 = p \in (0,1)\), we get that \(P\set{\tau_S^{\set{1}}= n} = (1-p)^{n-1}p\) for all \(n \in \N\). It follows that Hence, the random time \(\tau_S^{\set{1}}\) is finite almost surely. We will show that \(\tau_S^{\set{k}}\) is finite almost surely for all \(k \in \N\) by induction. By induction hypothesis, \(\tau_S^{\set{k-1}}\) is finite almost surely. Then \(S_{\tau_S^{\set{k-1}}+n}-S_{\tau_S^{\set{k-1}}} = \sum_{j=1}^nX_{\tau_S^{\set{k-1}}+j}\) is the sum of \(n\) Bernoulli random variables independent of \(\tau_S^{\set{k-1}}\) by strong independence property, and hence has distribution identical to \(S_n\). Further, This implies that \(\tau_S^{\set{k}} = \tau_S^{\set{k-1}} + \tau_S^{\set{1}}\), where \(\tau_S^{\set{1}}\) has the identical distribution to \(\tau_X^{\set{1}}\) and is independent of \(\tau_S^{\set{k-1}}\). ◻